<!DOCTYPE html>
<html>
<head><meta name="generator" content="Hexo 3.8.0">
  <meta charset="utf-8">
  
  <title>【翻译】阻塞IO、非阻塞IO、Epoll | gdme1320 的笔记</title>
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
  <meta name="description" content="Table of Contents   Intro 阻塞模式 非阻塞模式 (O_NONBLOCK) IO多路复用（select, epoll, kqueue等） O_NONBLOCK和I/O Multiplexing Edge-Triggered Polling 防失效原文     Intro    Blocking I/O, Nonblocking I/O, And Epoll    在这">
<meta property="og:type" content="article">
<meta property="og:title" content="【翻译】阻塞IO、非阻塞IO、Epoll">
<meta property="og:url" content="http://gdme1320.gitee.com/linux/non-blocing-io-epoll/index.html">
<meta property="og:site_name" content="gdme1320 的笔记">
<meta property="og:description" content="Table of Contents   Intro 阻塞模式 非阻塞模式 (O_NONBLOCK) IO多路复用（select, epoll, kqueue等） O_NONBLOCK和I/O Multiplexing Edge-Triggered Polling 防失效原文     Intro    Blocking I/O, Nonblocking I/O, And Epoll    在这">
<meta property="og:locale" content="default">
<meta property="og:updated_time" content="2022-03-23T09:59:24.248Z">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="【翻译】阻塞IO、非阻塞IO、Epoll">
<meta name="twitter:description" content="Table of Contents   Intro 阻塞模式 非阻塞模式 (O_NONBLOCK) IO多路复用（select, epoll, kqueue等） O_NONBLOCK和I/O Multiplexing Edge-Triggered Polling 防失效原文     Intro    Blocking I/O, Nonblocking I/O, And Epoll    在这">
  
    <link rel="alternate" href="/atom.xml" title="gdme1320 的笔记" type="application/atom+xml">
  
  
  
    <link href="//fonts.googleapis.com/css?family=Source+Code+Pro" rel="stylesheet" type="text/css">
  

  <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" crossorigin="anonymous">

  <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css" integrity="sha384-XdYbMnZ/QjLh6iI4ogqCTaIjrFk87ip+ekIjefZch0Y+PvJ8CDYtEs1ipDmPorQ+" crossorigin="anonymous">

  <link rel="stylesheet" href="/css/styles.css">
  <link rel="stylesheet" href="/css/docs.min.css">
  

</head>
</html>
<body>
  <nav class="navbar navbar-inverse">
  <div class="container">
    <!-- Brand and toggle get grouped for better mobile display -->
    <div class="navbar-header">
      <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#main-menu-navbar" aria-expanded="false">
        <span class="sr-only">Toggle navigation</span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
      </button>
      
    </div>

    <!-- Collect the nav links, forms, and other content for toggling -->
    <div class="collapse navbar-collapse" id="main-menu-navbar">
      <ul class="nav navbar-nav">
        
          <li><a class href="/index.html">Home</a></li>
        
          <li><a class href="/archives/">Archives</a></li>
        
      </ul>

      <!--
      <ul class="nav navbar-nav navbar-right">
        
          <li><a href="/atom.xml" title="RSS Feed"><i class="fa fa-rss"></i></a></li>
        
      </ul>
      -->
    </div><!-- /.navbar-collapse -->
  </div><!-- /.container-fluid -->
</nav>

  <div class="container bs-docs-container">
  
    <div class="row">
        <div class="col-sm-8 blog-main">
          <article id="post-linux/non-blocing-io-epoll" class="article article-type-post" itemscope itemprop="blogPost">

  <header class="article-header">
    
  
    <h1 class="article-title" itemprop="name">
      【翻译】阻塞IO、非阻塞IO、Epoll
    </h1>
  


  </header>

  <div class="article-meta">
    <div class="article-datetime">
  <a href="/linux/non-blocing-io-epoll/" class="article-date"><time datetime="2022-03-23T09:59:24.248Z" itemprop="datePublished">2022-03-23</time></a>
</div>

    
    
  <div class="article-category">
    <a class="article-category-link" href="/categories/linux/">linux</a>
  </div>


  </div>
  <div class="article-inner">

    <div class="article-entry" itemprop="articleBody">
      
        
<div id="content">
<div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#org4b84c78">Intro</a></li>
<li><a href="#org1ed3c86">阻塞模式</a></li>
<li><a href="#org8f3e2fb">非阻塞模式 (O_NONBLOCK)</a></li>
<li><a href="#org10f6003">IO多路复用（select, epoll, kqueue等）</a></li>
<li><a href="#org6015c01">O_NONBLOCK和I/O Multiplexing</a></li>
<li><a href="#org394f4d3">Edge-Triggered Polling</a></li>
<li><a href="#orgfe317d6">防失效原文</a></li>
</ul>
</div>
</div>
<div class="outline-2" id="outline-container-org4b84c78">
<h2 id="org4b84c78">Intro</h2>
<div class="outline-text-2" id="text-org4b84c78">
<blockquote>
<p>
<a href="https://eklitzke.org/blocking-io-nonblocking-io-and-epoll" target="_blank" rel="noopener">Blocking I/O, Nonblocking I/O, And Epoll</a>
</p>
</blockquote>
<p>
在这篇文章中，我想详细地解释当你使用  nonblocking IO  （非阻塞式IO）时所发生的事：包括如下内容：
</p>
<ul class="org-ul">
<li>使用  fcntl  设置  fd  （file descriptor，文件描述符）的  O_NONBLOCK  的语法。</li>
<li>nonblocking IO和   asynchronous I/O  （异步IO）的区别</li>
<li>为什么nonblocking IO经常与  IO multiplexers  （IO多路复用，例如  select   epoll   kqueue  等）一起使用。</li>
<li>非阻塞模式如何与epoll的  edge-triggered  polling（边缘触发）交互。</li>
</ul>
</div>
</div>
<div class="outline-2" id="outline-container-org1ed3c86">
<h2 id="org1ed3c86">阻塞模式</h2>
<div class="outline-text-2" id="text-org1ed3c86">
<p>
Unix系统上所有的fd默认都是阻塞模式。也就是说系统调用，像  read   write   connect  会阻塞。
通过一个简单的方式理解阻塞：想想你的控制台应用程序，从  stdin  读取用户输入的时候，你的程序就处于阻塞状态，直到有数据到达————例如用户在键盘上敲了几个字符。
在内部，系统内核将进程投入睡眠状态，直到stdin上有数据能读。
其他的fd同样如此。比如，如果你想从TCP Socket上读取数据， ~read~方法会一直阻塞，直到连接的另一端发送了数据过来。
</p>
<p>
对于并发应用程序来说，阻塞就是个问题了，因为阻塞的程序被挂起。有两种不同而互补的方式解决这个问题：
</p>
<ul class="org-ul">
<li>Nonblocking mode 非阻塞模式</li>
<li>I/O multiplexing IO多路复用，例如  select   epoll </li>
</ul>
<p>
这两种方式虽然经常一起使用，但它们是独立的策略，一会我们将看到它们的区别，以及为什么它们 <b>经常</b> 一起使用。
</p>
</div>
</div>
<div class="outline-2" id="outline-container-org8f3e2fb">
<h2 id="org8f3e2fb">非阻塞模式 (O_NONBLOCK)</h2>
<div class="outline-text-2" id="text-org8f3e2fb">
<p>
通过将  O_NONBLOCK  添加到  fcntl  的标志中，我们将fd变成非阻塞模式。
</p>
<div class="org-src-container">
<pre class="src src-c"><span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">set O_NONBLOCK on fd </span><span style="font-weight: bold; font-style: italic;">*/</span>
<span style="font-weight: bold; text-decoration: underline;">int</span> <span style="font-weight: bold; font-style: italic;">flags</span> = fcntl(fd, F_GETFL, 0); <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">fd 原来的flags </span><span style="font-weight: bold; font-style: italic;">*/</span>
<span style="font-weight: bold;">fcntl</span>(fd, F_SETFL, flags | O_NONBLOCK); <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">flags位开关，设置fd的flags </span><span style="font-weight: bold; font-style: italic;">*/</span>
</pre>
</div>
<p>
从现在开始，fd是当作非阻塞的了。  read   write  之类的会发生阻塞的IO系统调用，将返回-1，并且设置  errno=EWOULDBLOCK 
</p>
<p>
看起来很有意思，实际上却不是那么有用。仅这种原始的机制还无法有效实现IO多路复用。
例如，我们想一次从两个fd读取数据，于是我们写一个循环来检查每一个fd上有没有数据，然后在下次检查之前睡眠一段时间。
</p>
<div class="org-src-container">
<pre class="src src-c"><span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">两个fd：fd1, fd2 已经设置了非阻塞 </span><span style="font-weight: bold; font-style: italic;">*/</span>
<span style="font-weight: bold;">struct</span> <span style="font-weight: bold; text-decoration: underline;">timespec</span> <span style="font-weight: bold; font-style: italic;">sleep_interval</span>{.tv_sec = 0, .tv_nsec = 1000};
<span style="font-weight: bold; text-decoration: underline;">ssize_t</span> <span style="font-weight: bold; font-style: italic;">nbytes</span>;
<span style="font-weight: bold;">for</span> (;;) {
    <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">try fd1 </span><span style="font-weight: bold; font-style: italic;">*/</span>
    <span style="font-weight: bold;">if</span> ((nbytes = read(fd1, buf, <span style="font-weight: bold;">sizeof</span>(buf))) &lt; 0) {
        <span style="font-weight: bold;">if</span> (errno != EWOULDBLOCK) {
            perror(<span style="font-style: italic;">"read/fd1"</span>);
        }
    } <span style="font-weight: bold;">else</span> {
        handle_data(buf, nbytes);
    }

    <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">try fd2 </span><span style="font-weight: bold; font-style: italic;">*/</span>
    <span style="font-weight: bold;">if</span> ((nbytes = read(fd2, buf, <span style="font-weight: bold;">sizeof</span>(buf))) &lt; 0) {
        <span style="font-weight: bold;">if</span> (errno != EWOULDBLOCK) {
            perror(<span style="font-style: italic;">"read/fd2"</span>);
        }
    } <span style="font-weight: bold;">else</span> {
        handle_data(buf, nbytes);
    }

    <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">sleep for a bit; real version needs error checking! </span><span style="font-weight: bold; font-style: italic;">*/</span>
    nanosleep(sleep_interval, <span style="font-weight: bold; text-decoration: underline;">NULL</span>);
}
</pre>
</div>
<p>
这代码虽然能工作，但是有很多缺点：
</p>
<ul class="org-ul">
<li>当数据来得很慢时，程序被多次不必要唤醒，浪费CPU</li>
<li>数据到达时，程序还在睡眠中，这就造成了延迟。</li>
<li>用这种方式处理更多的fd是非常笨重的</li>
</ul>
<p>
要修复这种问题，我们需要IO多路复用
</p>
</div>
</div>
<div class="outline-2" id="outline-container-org10f6003">
<h2 id="org10f6003">IO多路复用（select, epoll, kqueue等）</h2>
<div class="outline-text-2" id="text-org10f6003">
<p>
有多个IO多路复用的系统调用：
</p>
<ul class="org-ul">
<li>select是POSIX中定义的</li>
<li>linux上的epoll</li>
<li>BSD上的kqueue</li>
</ul>
<p>
从根本上讲它们工作的方式是相同的：它们告诉内核我们在多个fd上关注的事件（典型的事件如read,write），然后它们阻塞直到我们关注的发生。
例如，你能够告诉内核你对fd X上的read事件感兴趣，Y上的read上write感兴趣，z上的write感兴趣。
</p>
<p>
这些IO多路复用的系统调用并不关心fd是阻塞的还是非阻塞的。你将所有的fd保留原来的阻塞模式，它们还能够与select epoll正常工作。
如果你在select epoll上返回的fd上只调用read write并不会阻塞。即使它们是阻塞模式的。
但是有一个非常重要的例外，fd上的阻塞还是非阻塞对于edge-triggered polling有着重要的意义，下面将解释。
</p>
<p>
IO多路复用的并发方式，我将它称为“异步IO”。有的人将它称为“非阻塞IO”，我觉得这是对系统编程层面上“非阻塞”的混淆。
我建议保留术语“nonbloking”作为“fd是阻塞模式还是非阻塞模式”。
</p>
</div>
</div>
<div class="outline-2" id="outline-container-org6015c01">
<h2 id="org6015c01">O_NONBLOCK和I/O Multiplexing</h2>
<div class="outline-text-2" id="text-org6015c01">
<p>
假设我们要写一个简单的socket服务器，使用阻塞fd和select。简单起见，这个例子中我们只有我们想读的fd（read_fd）。
下面是核心部分：调用select的事件循环和调用read
</p>
<div class="org-src-container">
<pre class="src src-c"><span style="font-weight: bold; text-decoration: underline;">ssize_t</span> <span style="font-weight: bold; font-style: italic;">nbytes</span>;
<span style="font-weight: bold;">for</span> (;;) {
    <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">select call happens here </span><span style="font-weight: bold; font-style: italic;">*/</span>
    <span style="font-weight: bold;">if</span> (select(FD_SETSIZE, &amp;read_fds, <span style="font-weight: bold; text-decoration: underline;">NULL</span>, <span style="font-weight: bold; text-decoration: underline;">NULL</span>, <span style="font-weight: bold; text-decoration: underline;">NULL</span>) &lt; 0) {
        perror(<span style="font-style: italic;">"select"</span>);
        exit(EXIT_FAILURE);
    }
    <span style="font-weight: bold;">for</span> (<span style="font-weight: bold; text-decoration: underline;">int</span> <span style="font-weight: bold; font-style: italic;">i</span> = 0; i &lt; FD_SETSIZE; i++) {
        <span style="font-weight: bold;">if</span> (FD_ISSET(i, &amp;read_fds)) {
            <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">read call happens here </span><span style="font-weight: bold; font-style: italic;">*/</span>
            <span style="font-weight: bold;">if</span> ((nbytes = read(i, buf, <span style="font-weight: bold;">sizeof</span>(buf))) &gt;= 0) {
                handle_read(nbytes, buf);
            } <span style="font-weight: bold;">else</span> {
                <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">real version needs to handle EINTR correctly </span><span style="font-weight: bold; font-style: italic;">*/</span>
                perror(<span style="font-style: italic;">"read"</span>);
                exit(EXIT_FAILURE);
            }
        }
    }
}
</pre>
</div>
<p>
似乎很完美。但是，如果buf很小，然后很多数据传过来了怎么办？假设，buf是1024byte，面64KB的数据一次到达。
为了处理这个请求，我们要调用select接着是read 64次，一共128次系统调用，这就非常多了。
</p>
<p>
如果buffer太小了，read就会被调用很多次，没有方法避免。但是也许我们能减少select的调用次数？理论上我们能只调用select一次。
</p>
<p>
事实上没问题，通过将fd设置为非阻塞模式。要点是在一次循环中一直调用read，直到read返回EWOULDBLOCK：
</p>
<div class="org-src-container">
<pre class="src src-c"><span style="font-weight: bold; text-decoration: underline;">ssize_t</span> <span style="font-weight: bold; font-style: italic;">nbytes</span>;
<span style="font-weight: bold;">for</span> (;;) {
    <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">select call happens here </span><span style="font-weight: bold; font-style: italic;">*/</span>
    <span style="font-weight: bold;">if</span> (select(FD_SETSIZE, &amp;read_fds, <span style="font-weight: bold; text-decoration: underline;">NULL</span>, <span style="font-weight: bold; text-decoration: underline;">NULL</span>, <span style="font-weight: bold; text-decoration: underline;">NULL</span>) &lt; 0) {
        perror(<span style="font-style: italic;">"select"</span>);
        exit(EXIT_FAILURE);
    }
    <span style="font-weight: bold;">for</span> (<span style="font-weight: bold; text-decoration: underline;">int</span> <span style="font-weight: bold; font-style: italic;">i</span> = 0; i &lt; FD_SETSIZE; i++) {
        <span style="font-weight: bold;">if</span> (FD_ISSET(i, &amp;read_fds)) {
            <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">NEW: loop until EWOULDBLOCK is encountered </span><span style="font-weight: bold; font-style: italic;">*/</span>
            <span style="font-weight: bold;">for</span> (;;) {
                <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">read call happens here </span><span style="font-weight: bold; font-style: italic;">*/</span>
                nbytes = read(i, buf, <span style="font-weight: bold;">sizeof</span>(buf));
                <span style="font-weight: bold;">if</span> (nbytes &gt;= 0) {
                    handle_read(nbytes, buf);
                } <span style="font-weight: bold;">else</span> {
                    <span style="font-weight: bold;">if</span> (errno != EWOULDBLOCK) {
                        <span style="font-weight: bold; font-style: italic;">/* </span><span style="font-weight: bold; font-style: italic;">real version needs to handle EINTR correctly </span><span style="font-weight: bold; font-style: italic;">*/</span>
                        perror(<span style="font-style: italic;">"read"</span>);
                        exit(EXIT_FAILURE);
                    }
                    <span style="font-weight: bold;">break</span>;
                }
            }
        }
    }
}
</pre>
</div>
<p>
上面的例子中（1024 bytes buffer, 64KB imcomming data），我们执行了66次系统调用：
select仅调用了一次，read无错误调用了64次，还有一次是返回了EWOULDBLOCK的调用。
这样就好多了，只有上面那个的一半，大幅提升了性能和扩展性。
</p>
<p>
这样做的缺点，由于多了个循环，至少有一次多余的read调用。
假如buffer足够大，能够读入所有数据，那么系统调用的次数是3次而不是2次：
</p>
<ol class="org-ol">
<li>select,</li>
<li>read所有，这个read并没有返回-1，而是已读取的字节数，因此还会进入一次循环，</li>
<li>这次的read才设置EWOULDBLOCK。循环结束。</li>
</ol>
</div>
</div>
<div class="outline-2" id="outline-container-org394f4d3">
<h2 id="org394f4d3">Edge-Triggered Polling</h2>
<div class="outline-text-2" id="text-org394f4d3">
<p>
使用Non-blocking IO还有一个非常重要的作用：边缘触发系统调用。这种系统有两种模式：
</p>
<ul class="org-ul">
<li>level-triggered polling，更加简单的编程模式，类似于经典的select系统调用。</li>
<li>edge-triggered polling</li>
</ul>
<p>
要理解它们的不同，我们需要理解内核中epoll如何工作。
</p>
<p>
假设你告诉内核你关注一些fd上读事件，系统为每个fd维护一个关注列表。当数据到达fd，系统浏览兴趣列表，然后唤醒被epoll_wait阻塞的进程。
</p>
<p>
上面的过程，无论epoll是哪种触发模式，都会改生。两种触发模式之间的区别，是当你调用epoll_wait时，内核中发作的事。
在level-triggered模式下，内核遍历兴趣列表中每个fd，看它是否符合兴趣的条件。
例如，如果你关注fd 8上的read事件，调用epoll_wait时，内核首先检查：8上是否有数据能读，如果任何fd满足条件，epoll_wait将返回而不阻塞。
</p>
<p>
而在edge-triggered模式下，当调用epoll_wait时，内核跳过检查，并且立即将进程投入睡眠。
这种模式下，你将承担所有的职责，正确处理，在进入等待之前每个fd都需要读写所有的数据。
</p>
<p>
edge-triggered使得epoll成为O(1) I/O的多路复用：epoll_wait将立即挂起，由于之前维护的列表，数据到达之后，内核在O(1)时间内立即知道哪个进程应该被唤醒。
</p>
<p>
两种模式还有其他区别。假如你的read buffer是100字节，当200字节数据到达，然后你调用一次read，随后调用epoll_wait。缓冲中还剩100字节能读。
在level-triggered模式下，内核能够查觉到，并通知进程应该再次调用read。
而在edge-triggered模式下，内核立即将进程投入睡眠。如果连接的另一端也期待着回复（例如它发送的数据是类似RPC），那这两边就死锁了：服务端等待客户端发送更多的数据，客户端等着服务端的回复。
</p>
<p>
要使用edge-triggered，你必须将fd设置成非阻塞模式。然后你需要调用read或write，直到返回EWOULDBLOCK。如果不满足这些条件，你将错过内核的通知。
这么做有一个好处：每次epoll_wait调用都会非常高效，尤其对于高并发程序非常重要。如果你想知道更多这个我强烈推荐的方法，请阅读<a href="http://man7.org/linux/man-pages/man7/epoll.7.html" target="_blank" rel="noopener">epoll(7) man page</a>。
</p>
<p>
完整的代码在<a href="https://github.com/eklitzke/epollet" target="_blank" rel="noopener">GitHub</a>。
</p>
</div>
</div>
<div class="outline-2" id="outline-container-orgfe317d6">
<h2 id="orgfe317d6">防失效原文</h2>
<div class="outline-text-2" id="text-orgfe317d6">
<p>
Blocking I/O, Nonblocking I/O, And Epoll
Jan 10, 2017
In this post I want to explain exactly what happens when you use nonblocking I/O. In particular, I want to explain:
The semantics of setting O_NONBLOCK on a file descriptor using fcntl
How nonblocking I/O is different from asynchronous I/O
Why nonblocking I/O is frequently used in conjunction with I/O multiplexers like select, epoll, and kqueue
How nonblocking mode interacts with edge-triggered polling with epoll
Blocking Mode
By default, all file descriptors on Unix systems start out in “blocking mode”. That means that I/O system calls like read, write, or connect can block. A really easy way to understand this is to think about what happens when you read data on stdin from a regular TTY-based program. If you call read on stdin then your program will block until data is actually available, such as when the user actually physically types characters on their keyboard. Specifically, the kernel will put the process into the “sleeping” state until data is available on stdin. This is also the case for other types of file descriptors. For instance, if you try to read from a TCP socket then the read call will block until the other side of the connection actually sends data.
Blocking is a problem for programs that should operate concurrently, since blocked processes are suspended. There are two different, complementary ways to solve this problem. They are:
Nonblocking mode
I/O multiplexing system calls, such as select and epoll
These two solutions are often used together, but they are independent strategies to solving this problem, and often both are used. In a moment we’ll see the difference and why they’re commonly both used.
Nonblocking Mode (O_NONBLOCK)
A file descriptor is put into “nonblocking mode” by adding O_NONBLOCK to the set of fcntl flags on the file descriptor:
<i>* set O_NONBLOCK on fd *</i>
int flags = fcntl(fd, F_GETFL, 0);
fcntl(fd, F_SETFL, flags | O_NONBLOCK);
From this point forward the file descriptor is considered nonblocking. When this happens I/O system calls like read and write that would block will return -1, and errno will be set to EWOULDBLOCK.
This is interesting, but on its own is actually not that useful. With just this primitive there’s no efficient way to do I/O on multiple file descriptors. For instance, suppose we have two file descriptors and want to read both of them at once. This could be accomplished by having a loop that checks each file descriptor for data, and then sleeps momentarily before checking again:
struct timespec sleep_interval{.tv_sec = 0, .tv_nsec = 1000};
ssize_t nbytes;
for (;;) {
    <i>* try fd1 *</i>
    if ((nbytes = read(fd1, buf, sizeof(buf))) &lt; 0) {
        if (errno != EWOULDBLOCK) {
            perror("read/fd1");
        }
    } else {
        handle_data(buf, nbytes);
    }
</p>
<p>
<i>* try fd2 *</i>
if ((nbytes = read(fd2, buf, sizeof(buf))) &lt; 0) {
    if (errno != EWOULDBLOCK) {
	perror("read/fd2");
    }
} else {
    handle_data(buf, nbytes);
}
</p>
<p>
<i>* sleep for a bit; real version needs error checking! *</i>
    nanosleep(sleep_interval, NULL);
}
This works, but has a lot of drawbacks:
When data is coming in very slowly the program will wake up frequently and unnecessarily, which wastes CPU resources.
When data does come in, the program may not read it immediately if it’s sleeping, so the latency of the program will be poor.
Handling a large number of file descriptors with this pattern would become cumbersome.
To fix these problems we need an I/O multiplexer.
I/O Multiplexing (select, epoll, kqueue, etc.)
There’s a few I/O multiplexing system calls. Examples of I/O multiplexing calls include select (defined by POSIX), the epoll family on Linux, and the kqueue family on BSD. These all work fundamentally the same way: they let the kernel know what events (typically read events and write events) are of interest on a set of file descriptors, and then they block until something of interest happens. For instance, you might tell the kernel you are interested in just read events on file descriptor X, both read and write events on file descriptor Y, and just write events on file descriptor Z.
These I/O multiplexing system calls typically do not care if the file descriptors are in blocking mode or nonblocking mode. You can leave all of your file descriptors in blocking mode and they’ll work just fine with select or epoll. If you only call read and write on file descriptors returned by select or epoll the calls won’t block, even if those file descriptors are in blocking mode. There’s one important exception! The blocking or nonblocking status of a file descriptor is significant for edge-triggered polling, as explained further below.
The multiplexing approach to concurrency is what I call “asynchronous I/O”. Sometimes people will call this same approach “nonblocking I/O”, which I believe comes from a confusion about what “nonblocking” means at the systems programming level. I suggest reserving the term “nonblocking” for referring to whether or not file descriptors are actually in nonblocking mode or not.
How O_NONBLOCK Interacts With I/O Multiplexing
Let’s say we’re writing a simple socket server using select with blocking file descriptors. For simplicity, in this example we just have file descriptors we want to read from, which are in read_fds. The core part of the event loop will call select and then invoke read once for each file descriptor with data:
ssize_t nbytes;
for (;;) {
    <i>* select call happens here *</i>
    if (select(FD_SETSIZE, &amp;read_fds, NULL, NULL, NULL) &lt; 0) {
        perror("select");
        exit(EXIT_FAILURE);
    }
    for (int i = 0; i &lt; FD_SETSIZE; i++) {
        if (FD_ISSET(i, &amp;read_fds)) {
            <i>* read call happens here *</i>
            if ((nbytes = read(i, buf, sizeof(buf))) &gt;= 0) {
                handle_read(nbytes, buf);
            } else {
                <i>* real version needs to handle EINTR correctly *</i>
                perror("read");
                exit(EXIT_FAILURE);
            }
        }
    }
}
This works and it’s perfectly fine. But, what happens if buf is small, and a lot of data comes down the line? To be concrete, suppose that buf is a 1024-byte buffer but 64KB of data comes in all at once. To handle this request we’ll invoke select followed by read 64 times. That’s 128 total system calls, which is a lot.
If the buffer size is too small read will have to be called a lot of times, there’s no avoiding that. But perhaps we could reduce the number of times we call select? Ideally in this example we would call select only one time.
In fact, this is possible, and it’s accomplished by putting the file descriptors into nonblocking mode. The basic idea is that you keep calling read in a loop until it returns EWOULDBLOCK. That looks like this:
ssize_t nbytes;
for (;;) {
    <i>* select call happens here *</i>
    if (select(FD_SETSIZE, &amp;read_fds, NULL, NULL, NULL) &lt; 0) {
        perror("select");
        exit(EXIT_FAILURE);
    }
    for (int i = 0; i &lt; FD_SETSIZE; i++) {
        if (FD_ISSET(i, &amp;read_fds)) {
            <i>* NEW: loop until EWOULDBLOCK is encountered *</i>
            for (;;) {
                <i>* read call happens here *</i>
                nbytes = read(i, buf, sizeof(buf));
                if (nbytes &gt;= 0) {
                    handle_read(nbytes, buf);
                } else {
                    if (errno != EWOULDBLOCK) {
                        <i>* real version needs to handle EINTR correctly *</i>
                        perror("read");
                        exit(EXIT_FAILURE);
                    }
                    break;
                }
            }
        }
    }
}
In this example (1024-byte buffer with 64KB of data incoming) we’ll do 66 system calls: select will be called one time, read will be called without error 64 times, and read will be called and return EWOULDBLOCK one time. This is much better! This is nearly half the number from the previous example, which will improve performance and scalability considerably.
The downside of this approach is that due to the new loop, there’s at least one extra read that happens since it’s called until it returns EWOULDBLOCK. Let’s say that typically the read buffer is large enough to read all of the incoming data in one read call. Then in the usual case through the loop there will be three system calls rather than just two: select to wait for the data, read to actually read the data, and then read again to get EWOULDBLOCK.
Edge-Triggered Polling
There’s one more important use of nonblocking I/O: with edge-triggered polling in the epoll system call. This system call has two modes: level-triggered polling, and edge-triggered polling. Level-triggered polling is a simpler programming model that’s similar to the classic select system call. To explain the difference we need to understand how epoll works within the kernel.
Suppose you tell the kernel you’re interested in using epoll to monitor read events on some file descriptor. The kernel maintains a list of these interests for each file descriptor. When data comes in on the file descriptor the kernel traverses the interests list and wakes up each process that was blocked in epoll_wait with that file descriptor in the event list.
What I outlined above happens regardless of what triggering mode epoll is in. The difference between level-triggered and edge-triggered polling is what happens in the kernel when you call epoll_wait. In level-triggered mode the kernel will traverse each file descriptor in the interest list to see if it already matches the interest condition. For instance, if you registered a read event on file descriptor 8, when calling epoll_wait the kernel will first check: does file descriptor 8 already have data ready for reading? If any of the file descriptors match the interest then epoll_wait can return without blocking.
By contrast, in edge-triggered mode the kernel skips this check and immediately puts the process to sleep when it calls epoll_wait. This puts all of the responsibility on you, the programmer, to do the Right Thing and fully read and write all data for each file descriptor before waiting on this.
This edge-triggered mode is what makes epoll an O(1) I/O multiplexer: the epoll_wait call will suspend immediately, and since a list is maintained for each file descriptor ahead of time, when new data comes in the kernel immediately knows what processes must be woken up in O(1) time.
Here’s a more worked out example of the difference between edge-triggered and level-triggered modes. Suppose your read buffer is 100 bytes, and 200 bytes of data comes in for that file descriptor. Suppose then you call read exactly one time and then call epoll_wait again. There’s still 100 bytes of data already ready to read. In level-triggered mode the kernel would notice this and notify the process that it should call read again. By contrast, in edge-triggered mode the kernel would immediately go to sleep. If the other side is expecting a response (e.g. the data it sent is some kind of RPC) then the two sides will “deadlock”, as the server will be waiting for the client to send more data, but the client will be waiting for the server to send a response.
To use edge-triggered polling you must put the file descriptors into nonblocking mode. Then you must call read or write until they return EWOULDBLOCK every time. If you fail to meet these conditions you will miss notifications from the kernel. But there’s a big upside of doing this: each call to epoll_wait will be more efficient, which can be very important on programs with extremely high levels of concurrency. If you want to learn more about the details I strongly encourage you to read the epoll(7) man page.
Update: I’ve put up a complete example of using edge-triggered epoll on GitHub: <a href="https://github.com/eklitzke/epollet" target="_blank" rel="noopener">https://github.com/eklitzke/epollet</a>
</p>
</div>
</div>
</div>
<div class="status" id="postamble">
<p class="author">Author: gdme1320</p>
<p class="validation"><a href="http://validator.w3.org/check?uri=referer" target="_blank" rel="noopener">Validate</a></p>
</div>

      
    </div>

    
      

    

    <footer class="article-footer">
      <a data-url="http://gdme1320.gitee.com/linux/non-blocing-io-epoll/" data-id="cl13eo7mc00dyhjv4xqfampd7" class="article-share-link">
        <i class="fa fa-share"></i> Share
      </a>
      
      

    </footer>
  </div>
  
    
<ul id="article-nav" class="nav nav-pills nav-justified">
  
  <li role="presentation">
    <a href="/linux/wget/" id="article-nav-older" class="article-nav-link-wrap">
      <i class="fa fa-chevron-left pull-left"></i>
      <span class="article-nav-link-title">Wget笔记</span>
    </a>
  </li>
  
  
  <li role="presentation">
    <a href="/linux/Setup-Local-Repository-Apt/" id="article-nav-newer" class="article-nav-link-wrap">
      <span class="article-nav-link-title">搭建本地Ubuntu/Debian软件包仓库</span>
      <i class="fa fa-chevron-right pull-right"></i>
    </a>
  </li>
  
</ul>


  
</article>




        </div>
        <div class="col-sm-3 col-sm-offset-1 blog-sidebar">
          
  
  <div class="sidebar-module">
    <h4>Categories</h4>
    <ul class="sidebar-module-list"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/apps/">apps</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/architecture/">architecture</a><span class="sidebar-module-list-count">2</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/devops/">devops</a><span class="sidebar-module-list-count">10</span><ul class="sidebar-module-list-child"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/devops/kubectl/">kubectl</a><span class="sidebar-module-list-count">2</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/devops/kubernetes/">kubernetes</a><span class="sidebar-module-list-count">3</span></li></ul></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/dotnet/">dotnet</a><span class="sidebar-module-list-count">3</span><ul class="sidebar-module-list-child"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/dotnet/entityframework/">entityframework</a><span class="sidebar-module-list-count">1</span></li></ul></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/elasticsearch/">elasticsearch</a><span class="sidebar-module-list-count">4</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/emacs/">emacs</a><span class="sidebar-module-list-count">4</span><ul class="sidebar-module-list-child"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/emacs/org-mode/">org-mode</a><span class="sidebar-module-list-count">3</span></li></ul></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/git/">git</a><span class="sidebar-module-list-count">2</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/graphql/">graphql</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/groovy/">groovy</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/">java</a><span class="sidebar-module-list-count">52</span><ul class="sidebar-module-list-child"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/concurrent/">concurrent</a><span class="sidebar-module-list-count">4</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/jackson/">jackson</a><span class="sidebar-module-list-count">2</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/shiro/">shiro</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/spring/">spring</a><span class="sidebar-module-list-count">11</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/spring-amqp/">spring-amqp</a><span class="sidebar-module-list-count">5</span><ul class="sidebar-module-list-child"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/spring-amqp/spring-jpa/">spring-jpa</a><span class="sidebar-module-list-count">1</span></li></ul></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/spring-jpa/">spring-jpa</a><span class="sidebar-module-list-count">10</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/spring-security/">spring-security</a><span class="sidebar-module-list-count">3</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/thymeleaf/">thymeleaf</a><span class="sidebar-module-list-count">2</span><ul class="sidebar-module-list-child"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/java/thymeleaf/maven/">maven</a><span class="sidebar-module-list-count">1</span></li></ul></li></ul></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/jenkins/">jenkins</a><span class="sidebar-module-list-count">3</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/journal/">journal</a><span class="sidebar-module-list-count">7</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/js/">js</a><span class="sidebar-module-list-count">3</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/linux/">linux</a><span class="sidebar-module-list-count">35</span><ul class="sidebar-module-list-child"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/linux/kde/">kde</a><span class="sidebar-module-list-count">2</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/linux/kross/">kross</a><span class="sidebar-module-list-count">3</span><ul class="sidebar-module-list-child"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/linux/kross/ktorrent-scripts/">ktorrent-scripts</a><span class="sidebar-module-list-count">1</span></li></ul></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/linux/shell/">shell</a><span class="sidebar-module-list-count">6</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/linux/tcp/">tcp</a><span class="sidebar-module-list-count">1</span></li></ul></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/lisp/">lisp</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/mongodb/">mongodb</a><span class="sidebar-module-list-count">2</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/mssql/">mssql</a><span class="sidebar-module-list-count">3</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/mysql/">mysql</a><span class="sidebar-module-list-count">12</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/nginx/">nginx</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/node/">node</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/postgresql/">postgresql</a><span class="sidebar-module-list-count">2</span><ul class="sidebar-module-list-child"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/postgresql/client/">client</a><span class="sidebar-module-list-count">1</span></li></ul></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/python/">python</a><span class="sidebar-module-list-count">19</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/regex/">regex</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/rfc/">rfc</a><span class="sidebar-module-list-count">2</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/vim/">vim</a><span class="sidebar-module-list-count">3</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/categories/windows/">windows</a><span class="sidebar-module-list-count">7</span></li></ul>
  </div>



  
  <div class="sidebar-module">
    <h4>Tags</h4>
    <ul class="sidebar-module-list"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/tags/ReentrantLock/">ReentrantLock</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/tags/nohup/">nohup</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/tags/pandoc/">pandoc</a><span class="sidebar-module-list-count">1</span></li></ul>
  </div>



  
  <div class="sidebar-module">
    <h4>Tag Cloud</h4>
    <p class="tagcloud">
      <a href="/tags/ReentrantLock/" style="font-size: 10px;">ReentrantLock</a> <a href="/tags/nohup/" style="font-size: 10px;">nohup</a> <a href="/tags/pandoc/" style="font-size: 10px;">pandoc</a>
    </p>
  </div>


  
  <div class="sidebar-module">
    <h4>Archives</h4>
    <ul class="sidebar-module-list"><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2919/06/">June 2919</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2022/03/">March 2022</a><span class="sidebar-module-list-count">58</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2022/01/">January 2022</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2021/11/">November 2021</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2021/10/">October 2021</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2021/04/">April 2021</a><span class="sidebar-module-list-count">2</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2021/03/">March 2021</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2021/02/">February 2021</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2021/01/">January 2021</a><span class="sidebar-module-list-count">2</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2020/12/">December 2020</a><span class="sidebar-module-list-count">2</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2020/11/">November 2020</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2020/10/">October 2020</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2020/09/">September 2020</a><span class="sidebar-module-list-count">3</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2020/07/">July 2020</a><span class="sidebar-module-list-count">3</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2020/06/">June 2020</a><span class="sidebar-module-list-count">10</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2020/04/">April 2020</a><span class="sidebar-module-list-count">4</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2020/03/">March 2020</a><span class="sidebar-module-list-count">8</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2020/02/">February 2020</a><span class="sidebar-module-list-count">4</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2020/01/">January 2020</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/12/">December 2019</a><span class="sidebar-module-list-count">7</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/11/">November 2019</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/10/">October 2019</a><span class="sidebar-module-list-count">11</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/09/">September 2019</a><span class="sidebar-module-list-count">6</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/08/">August 2019</a><span class="sidebar-module-list-count">4</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/07/">July 2019</a><span class="sidebar-module-list-count">10</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/06/">June 2019</a><span class="sidebar-module-list-count">3</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/05/">May 2019</a><span class="sidebar-module-list-count">3</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/04/">April 2019</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/03/">March 2019</a><span class="sidebar-module-list-count">8</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/02/">February 2019</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2019/01/">January 2019</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2018/11/">November 2018</a><span class="sidebar-module-list-count">4</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2018/10/">October 2018</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2018/08/">August 2018</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2018/07/">July 2018</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2018/06/">June 2018</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2018/03/">March 2018</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2017/07/">July 2017</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2017/06/">June 2017</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2017/04/">April 2017</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/2015/05/">May 2015</a><span class="sidebar-module-list-count">1</span></li><li class="sidebar-module-list-item"><a class="sidebar-module-list-link" href="/archives/1970/01/">January 1970</a><span class="sidebar-module-list-count">9</span></li></ul>
  </div>



  
  <div class="sidebar-module">
    <h4>Recents</h4>
    <ul class="sidebar-module-list">
      
        <li>
          <a href="/java/acessing-generic-types-at-runtime/">获取泛型类型</a>
        </li>
      
        <li>
          <a href="/linux/shell/bash_cgi/">Script for Shell Parsing Parameters</a>
        </li>
      
        <li>
          <a href="/linux/shell/shell-script/">Shell Script Getting Start</a>
        </li>
      
        <li>
          <a href="/linux/tcp/network-tcp-performance-turning-with-sysctl/">【翻译】Sysctl调节Linux网络性能</a>
        </li>
      
        <li>
          <a href="/java/concurrent/aqs-src/">AQS源码笔记</a>
        </li>
      
    </ul>
  </div>



        </div>
    </div>
  </div>
  <footer class="blog-footer">
  <div class="container">
    <div id="footer-info" class="inner">
      &copy; 2022 gdme1320<br>
      Powered by <a href="http://hexo.io/" target="_blank">Hexo</a>
    </div>
  </div>
</footer>

  

<!-- <script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.4/jquery.min.js" integrity="sha384-8gBf6Y4YYq7Jx97PIqmTwLPin4hxIzQw5aDmUg/DDhul9fFpbbLcLh3nTIIDJKhx" crossorigin="anonymous"></script> -->

<script src="/js/jquery.min.js"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js" integrity="sha384-0mSbJDEHialfmuBBQP6A4Qrprq5OVfW37PRR3j5ELqxss1yVqOtnepnHVP9aJ7xS" crossorigin="anonymous"></script>



<script src="/js/script.js"></script>

</body>
</html>
